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DETAILED ACTION 
Specification 

1 . The disclosure is objected to because of the following informalities: 

a. On page 2, line 20 "425 millions" should be changed to -425 million-. 

b. On page 3, the end of line 5 needs a period. 

c. On page 4, line 6 "identify the his" should be changed to -identify his-. 

d. On page 14, line 5 "affect it's the" should be changed to -affect the-. 

e. On page 16, line 18 "keywords that helps" should be changed to -keyword 
that helps-. 

f. On page 17, line 4 a closed parenthesis is needed after FIG. 5B. 
Appropriate correction is required. 

2. The specification is objected to as failing to provide proper antecedent basis for 
the claimed subject matter. See 37 CFR 1 .75(d)(1 ) and MPEP § 608.01 (o). Correction 
of the following is required: Claims 27 and 28 mention types of data and comparison 
techniques, respectively, that are not mentioned or explained in the specification. 

* 

Claim Objections 

3. Claims 20 and 21 are objected to because of the following informalities: both 
claims refer to "test", which should be changed to text. 

t 

Appropriate correction is required. 
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Claim Rejections - 35 USC § 102 

■ 

4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

Claims 1-3, 5, 6, 9-14, 17, 18 and 25-28 are rejected under 35 U.S.C. 102(b) as 
being anticipated by Noda et al. (Cosmos: Convenient Image Retrieval System of 
Flowers for Mobile Computing Situations). 

Consider claim 1, Noda et al. explicitly teach: 

A mobile deixis device (Figure 1) comprising: 

(a) a camera to capture an image (digital camera and PDA, Figure 1); 

(b) a wireless communication device (PHS, Figure 1), coupled to the camera and 
to a wireless network (WWW server, Figure 1), to communicate the image with existing 
databases to find similar images (Flower Database, Figure 1); and 

(c) a processor (image retrieval, Figure 1 ), coupled to the wireless 
communication device, to process found database records related to similar images 
(pages 26 and 27, 3-3.4). 

Consider claim 2, Noda et al. explicitly teach: 

The mobile deixis device as recited in Claim 1 comprising a display to view found 
database records and the found database records comprising web pages including 
images (PDA browser, Figure 1, Figure 5, page 28, 3.5). 



r 
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Consider claim 3, Noda et al. explicitly teach: 

The mobile deixis device as recited in Claim 1 comprising: 

(a) a storage medium coupled to the processor (in order to perform the 
operations on page 26 and 27, 3-3.4 and extracting characteristics in Figure 2, it is 
inherent that there must be a memory and or storage medium); and 

(b) a plurality of programs stored (it is inherent that these processing commands 
must be stored, page 25, 2 describes the operation of the invention, which includes 
control of the Cosmos device) in the storage medium operative to interact with the 
processor and the wireless communication device to control the operation of the mobile 
deixis device, comprising: 

(i) a first program of the plurality of programs stored on the storage 
medium being operative to interact with the processor to capture the image from 
the camera (first we photograph a flower with a digital camera, then we send the 
image to a server, see page 25, 2); 

(ii) a second program of the plurality of programs stored on the storage 
medium being operative to interact with the processor to communicate with at 
least one database to find a similar image similar to the captured image (sent 
characteristics and the extracted characteristics are used in order to retrieve 
objective images of flowers, see page 25, 2, these images are retrieved from the 
flower database as seen in Figure 1 ); and 

(iii) a third program of the plurality of programs stored on the storage 
medium being operative to interact with the processor to provide to a display a 
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plurality of similar images and maintaining an associated hyperlink for each 

similar image (and the result of the retrieval is displayed on a web page 

constructed server. We look the page by a browser on the PDA and confirm the 

results, page 25, 2.1 , see also page 28, 3.5 and Figure 5). 

Consider claim 5, Noda et al. explicitly teach: 

The mobile deixis device as recited in Claim 1 comprising: 

(a) a storage medium coupled to the processor (in order to perform the 
operations on page 26 and 27, 3-3.4 and extracting characteristics in Figure 2, it is 
inherent that there must be a memory and or storage medium); and 

(b) a plurality of programs stored (it is inherent that these processing commands 
must be stored, page 25, 2 describes the operation of the invention, which includes 
control of the Cosmos device) in the storage medium operative to interact with the 
processor and the wireless communication device to control the operation of the mobile 
deixis device, comprising: 

(i) a first program of the plurality of programs stored on the storage 
medium being operative to interact with the processor to capture the image from 
the camera (first we photograph a flower with a digital camera, then we send the 

* 

image to a server, see page 25, 2); 

(ii) a second program of the plurality of programs stored on the storage 
medium being operative to interact with the processor to communicate with at 
least one computer having a database of images to cause the at least one 
computer to search the database of images to find a similar image similar to the 
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captured image (sent characteristics and the extracted characteristics are used in 

« 

order to retrieve objective images of flowers, see page 25, 2, these images are 
retrieved from the flower database as seen in Figure 1, database is interpreted 
as computer in that it has input, output and has to process in order to retrieve 
images from CD-ROMs mentioned on page 28, 4); and 

(iii) a third program of the plurality of programs stored on the storage 
medium being operative to interact with the processor to provide to a display of a 
plurality of similar images and maintaining an associated hyperlink for each 
similar image (and the result of the retrieval is displayed on a web page 
constructed server. We look the page by a browser on the PDA and confirm the 
results, see also page 28, 3.5 and Figure 5). 
Consider claim 6, Noda et al. explicitly teach: 
The mobile deixis device as recited in Claim 1 comprising: 

(a) a storage medium coupled to the processor (in order to perform the 
operations on page 26 and 27, 3-3.4 and extracting characteristics in Figure 2, it is 
inherent that there must be a memory and or storage medium); and 

(b) a plurality of programs stored (it is inherent that these processing commands 
must be stored, page 25, 2 describes the operation of the invention, which includes 
control of the Cosmos device) in the storage medium operative to interact with the 
processor and the wireless communication device to control the operation of the mobile 
deixis device, comprising: 
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(i) a first program of the plurality of programs stored on the storage 
medium being operative to interact with the processor to capture the image from 
the camera (first we photograph a flower with a digital camera, then we send the 
image to a server, see page 25, 2); 

(ii) a second program of the plurality of programs stored on the storage 
medium being operative to interact with the processor to communicate with at 
least one computer with web pages having images to search the images to find a 
similar image similar to the captured image (sent characteristics and the 
extracted characteristics are used in order to retrieve objective images of flowers, 
see page 25, 2, these images are retrieved from the flower database as seen in 
Figure 1 , database is interpreted as computer in that it has input, output and has 
to process in order to retrieve images from CD-ROMs mentioned on page 28, 4, 
Figure 5 demonstrates the web pages that are returned from the computer); and 

(iii) a third program of the plurality of programs stored on the storage 
medium being operative to interact with the processor to provide to a display of a 
plurality of similar images and maintaining an associated hyperlink for each 
similar image (and the result of the retrieval is displayed on a web page 
constructed server. We look the page by a browser on the PDA and confirm the 
results, see also page 28, 3.5 and Figure 5). 

Consider claim 9, Noda et al. explicitly teach: 

A system to provide location awareness services comprising: 

(a) a handheld device (digital camera, PDA, PHS, Figure 1) comprising: 
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(i) a camera to capture an image of an location (digital camera, Figure 1); 

(ii) a mobile communication device (PHS, Figure 1 ), coupled to the 
camera and to a wireless network (WWW sever, Figure 1 ), to 
communicate the image with existing computer files to find similar images 
(page 25, abstract, lines 8-12); and 

(iii) a user interface (PDA, Figure 1), coupled to the mobile communication 
device, to communicate to an user any results of found similar images 
(page 28, 3.5 and Figure 5); and 

(b) a computer network including the wireless network (WWW server, Figure 1 ) 
and a wired network (flower database, Figure 2); 

(c) a plurality of computers (WWW server and flower database, Figure 1 ), each 
computer have a plurality of computer files and connected to the computer network 
(process information files and flower information files respectively), at least one of the 
computer files having an image similar to the captured image and when viewed includes 
associated text describing an object in the image (page 26, paragraph 3, lines 8-13, see 

also Figure 5). 

Consider claim 10, Noda et al explicitly teach: 

The system to provide location awareness services as recited in Claim 9 wherein 
the user interface comprises a display (PDA, browser Figure 1 , see also Figure 5). 
Consider claim 11, Noda et al explicitly teach: 
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The system to provide location awareness services as recited in Claim 9 wherein 
the user interface comprises a computer connection (PDA connects to computers 
wirelessly through PHS, Figure 1). 

Consider claim 12, Noda et al. explicitly teach: 

The system to provide location awareness services as recited in Claim 9 wherein 
the associated text describes features of the object in the image including geographical 
location of the object (page 28, 3.5). 

Consider claim 13, Noda et al. explicitly teach: 

The system to provide location awareness services as recited in Claim 9 wherein 
the associated text describes features of the object in the image including a description 
and historical facts regarding the object (page 28, 3.5). 

t 

Consider claim 14, Noda et al. explicitly teach: 

The system to provide location awareness services as recited in Claim 9 wherein 
the associated text includes a uniform resource locator (URL) (Figure 5, the word detail 
is considered the URL because when it is clicked the web address corresponding to the 
link is opened and seen in Figure 6). 

Consider claim 17, Noda et al. explicitly teach: 

The system to provide location awareness services as recited in Claim 9 wherein 
at least one of the computers includes at least one computer file having an image of an 
object of known interest and an associated image of an object of less recognized 
interest within a predetermined radius about a geographical location of the known 
interest object (section 3.1 describes the extraction of the flower, which is the isolation 
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of the image and at the same time the isolation of the surroundings, therefore it is 
inherent that the surroundings can be used to identify an object of less interest using the 
same methods describe in the experimentation with real images 4.1) to aid a user in 
finding the object of less recognized interest. 

Consider claim 18, Noda et al. explicitly teach: 

The system to provide location awareness services as recited in Claim 9 wherein 
at least one of the computers includes at least one computer file having an image of an 
object of known interest and an associated image of an object of less recognized 
interest within the field of view of the known interest object (section 3.1 describes the 
extraction of the flower, which is the isolation of the image and at the same time the 
isolation of the surroundings, therefore it is inherent that the surroundings can be used 

♦ 

to identify an object of less interest using the same methods describe in the 
experimentation with real images 4.1 ) to aid a user in finding the object of less 
recognized interest. 

Consider claim 25, Noda et al. explicitly teach: 

A storage medium (in order to perform the operations on page 26 and 27, 3-3.4 
and extracting characteristics in Figure 2, it is inherent that there must be a memory and 
or storage medium) comprising: 

a first computer readable program code stored on the storage medium being 

* 

operative to interact with a processor in a handheld device to capture an image from a 
camera (first we photograph a flower with a digital camera, then we send the image to a 
server, which contains the memory, see page 25, 2); 
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a second computer readable program code stored on the storage medium being 
operative to interact with the processor to communicate with multiple data files and to 
search said data files for a similar image similar to the captured image (sent 
characteristics and the extracted characteristics are used in order to retrieve objective 
images of flowers, see page 25, 2, these images are retrieved from the flower database 
as seen in Figure 1 , database is interpreted as computer in that it has input, output and 
has to process in order to retrieve images from CD-ROMs mentioned on page 28, 4); 
and 

a third computer readable program code stored on the storage medium being 
operative to interact with the processor to provide to an user interface a plurality of 
similar images and maintaining an associated hyperlink for each similar image (and the 
result of the retrieval is displayed on a web page constructed server. We look the page 
by a browser on the PDA and confirm the results, see also page 28, 3.5 and Figure 5). 

Consider claim 26, Noda et al. explicitly teach: 

A method for identifying a location comprising the steps of: 

(i) providing a database of images, each image having an associated URL that 
includes said image and a description of the image (flower database, Figure 1 , see 
display in Figure 5, and page 28, 3.5); 

(ii) comparing an image of an unknown location with images from the database of 
images and providing a list of possible matching images(comparing characteristics, 
Figure 2); and 
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(iii) reviewing the images in the list of possible matching images until the correct 
location is identified (page 25, 2.1 , paragraph 3). 

Consider claim 27, Noda et al. explicitly teach: 

The method for identifying a location as recited in Claim 26 wherein the 
comparing step includes comparing at least one of energy spectrum data, color 
histogram data, primitive filter data, and local invariant data (pages 26-38, 3-3.4 
mentions color characteristic extraction). 

Consider claim 28, Noda et al. explicitly teach: 

The method for identifying a location as recited in claim 26 wherein the 
comparing step comprises at least one of the techniques including a least square 
matching technique, a normalizing the image technique, an eigen value technique, a 
matching histogram of image feature technique and an image matching engine with 
transformation technique (pages 27-28, 3.4). 

Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 
USPQ 459 (1966), that are applied for establishing a background for determining 
obviousness under 35 U.S.C. 103(a) are summarized as follows: 
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1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 

6 Claims 4, 7, 15, 16, 19, 20, 21 u and 24 are rejected under 35 U.S.C. 103(a) as 

being unpatentable over Noda et al. in view of Aarnio (US 6,522,889). 

Consider claim 4, Noda et al. explicitly teach: 

The mobile deixis device as recited in claim 3, including a second program being 
operative to interact with the processor to communicate with at least one server 
database (sent characteristics and the extracted characteristics are used in order to 
retrieve objective images of flowers, see page 25, 2, these images are retrieved from 
the flower database as seen in Figure 1 ). 

However, Noda et al. do not explicitly teach that the database can search further 
databases for a similar image similar to the captured image. 

In the same field of endeavor, Aarnio teaches a method and apparatus for 
providing precise location information through a communications network (Figures 1 
and 3). Aarnio further teaches that the database to be accessed is a computer network 
18, which may be the Internet, or World Wide Web (column 2, lines 57-60), which 
includes several databases that can then be checked. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to include the computer network found in Aarnio as the 
database in Noda et al. in order to pinpoint an exact location of the image by using the 
vast sources of the internet (column 1 , lines 35-39). 
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Consider claim 7, Noda et al. explicitly teach: 

The mobile deixis device as recited in Claim 3 wherein the at least one database 
includes a web site dispersed within the Internet (page 25, 2.1 , end of paragraph 2) and 
a plurality of programs stored in the storage medium (it is inherent that these processing 
commands must be stored, page 25, 2 describes the operation of the invention, which 
includes control of the Cosmos device). 

However, Noda et al. do not explicitly teach a program being operative to interact 
with the processor to identify any keywords linked to each similar image and a program 
being operative to interact with the processor to initiate a further search using the 
keywords to find additional similar images. 

In the same field of endeavor, Aarnio teaches a method and apparatus for 
providing precise location information through a communications network (Figures 1 
and 3). Aarnio further teaches a processor having a program in which digital image 
information can be searched directly or can be converted to binary text and these words 
can be further searched on the internet in order to find a location in accordance with the 
captured image (column 3, lines 50-67 through column 4, lines 1-27). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to include the text searching on the internet with words 
generated from the images found in Aarnio into the programs stored in the memory 
found in Noda et al. in order to enable the user to access location information such as 
street maps, locations of buildings, landmarks, etc. (column 4, lines 6-8). 

Consider claim 15, Noda et al explicitly teach: 
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The system to provide location awareness services as recited in Claim 9, 
wherein at least one of the computers includes a plurality of computer files having 
images of fields of interest (flower database Figure 1). 

However, Noda et al. do not explicitly teach that the database has information of 
locations of interest located within a predetermined radius about a geographical 

t 

location. 

In the came field of endeavor, Aarnio teaches a method and apparatus for 
providing precise location information through a communications network (Figures 1 
and 3). Aarnio further teaches a server service 24 that provides information to the user 
about the surroundings in the geographical region (column 4, lines 33-38). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to include the information about the geographical 
surroundings found in Aarnio into the images found in the database of Noda et al. in 
order to provide a convenience for the user such as in a case described in column 4, 
lines 38-43. 

Consider claim 16, the combination of Noda et al. in view of Aarnio, as applied to 

» 

claim 1 5, further teaches the system to provide location awareness services as recited 
in Claim 15 wherein the computer having a plurality of computer files having images of 
objects of interest located within a predetermined radius about a geographical location 
(column 4, lines 33-38, Aarnio) was previously trained to find common objects known to 
be of interest (page 26-28, 3-3.5 explains the finding of common images). 
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Consider claim 19, Noda et al explicitly teach the system to provide location 
awareness services as recited in Claim 9, which includes at least one computer (flower 

database Figure 1 ). 

However, Noda et al. do not teach that the computer has the capability of 
searching other computers to find matching files. 

In the same field of endeavor, Aarnio teaches a method and apparatus for 
providing precise location information through a communications network (Figures 1 
and 3). Aarnio further teaches that the database to be accessed is a computer network 
18, which may be the Internet, or World Wide Web (column 2, lines 57-60), which 
includes several computers that can then be checked. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to include the computer network found in Aarnio as the 
database in Noda et al. in order to pinpoint an exact location of the image by using the 
vast sources of the internet (column 1, lines 35-39). 

Consider claim 20, the combination of Noda et al. in view of Aarnio, as applied to 
claim 1 9, further teaches that the text that is found describing a matching image is 
communicated to the handheld device (page 25, 2.1, end of second paragraph). 

Consider claim 21, the combination of Noda et al. in view of Aarnio, as applied to 
claim 19, further teaches that the text is returned to the computer that began the search 
(Figure 4, in Aarnio, shows that the information that is found is returned to the mobile 
system, which is the computer that began the search). 
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Consider claim 22, the combination of Noda et al. in view of Aarnio, as applied to 
claim 21 above further teaches the system to provide location awareness services as 
recited in Claim 21 wherein the computer that initiated the search is capable of 
comparing the original image with images returned in the computer file having text 
matching the associated text describing the object in the image (see figures 5 and 6, the 
PDA provides the user with the comparison of images that resulted from the original 
computer, database, to the image). 

Consider claim 24, the combination of Noda et al. in view of Aarnio, as applied to 
claim 19, further teaches the system to provide location awareness services as recited 
in Claim 1 9 wherein at least one of the computers includes at least one computer file 
having an image of an object of known interest and an associated image of an object of 
less recognized interest within the field of view of the known interest object (section 3.1 
describes the extraction of the flower, which is the isolation of the image and at the 
same time the isolation of the surroundings, therefore it is inherent that the surroundings 
can be used to identify an object of less interest using the same methods describe in 
the experimentation with real images 4.1 ) to aid a user in finding the object of less 
recognized interest all located within a predetermined radius about a geographical 
location. 

7. Claim 8 is rejected under 35 U.S.C. 103(a) as being unpatentable over Noda et 
al. in view of Reed (US/2005/01 131 13). 

Consider claim 8, Noda et al. explicitly teach the mobile deixis device as recited 
in claim 3 including programs stored in the storage medium operative to interact with the 
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processor and the mobile communication device that are known to eliminated images 
based on comparison with other images, non matches. 

However, Noda et al. do not explicitly teach that the device includes a GPS 
system to identify the geographical location of the deixis device and be used to 
eliminate certain matches found. 

In the same field of endeavor, Reed teaches an enhance wireless phone (Figure 
2). Reed further teaches that this wireless phone has a GPS system 26, which is 
described in paragraph [01 14]. The GPS system locates the user and inherently would 
provide coordinates that could narrow down the image matching process. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to include the GPS capabilities of the mobile phone found 
in Reed into the Cosmos device found in Noda et al. in order to pinpoint exact locations 
and provide the user with the ease of mind of always knowing their location. 
8. Claim 23 is rejected under 35 U.S.C. 103(a) as being unpatentable over Noda et 
al. in view of Aarnio as applied to claim 15 above, and further in view of Reed. 

Consider claim 23, the combination of Noda et al. in view of Aarnio teaches the 
system to provide location awareness services as recited in claim 1 5. 

However, the combination does not explicitly teach that the system includes a 
GPS system receiver to identify the geographical location of the mobile communication 
device and help eliminate non-useful images. 

In the same field of endeavor, Reed teaches an enhance wireless phone (Figure 
2). Reed further teaches that this wireless phone has a GPS system 26, which is 
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described in paragraph [01 14]. The GPS system locates the user and inherently would 
provide coordinates that could narrow down the image matching process. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to include the GPS capabilities of the mobile phone found 
in Reed into the system found in the combination of Noda et al. in view of Aarnio in 
order to pinpoint exact locations and provide the user with the ease of mind of always 
knowing their location. 

Conclusion 

9. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. Lev et al. (US 2002/0102966) teaches and object identification 
method for a portable device, which takes a picture, sends the image to a server, which 
identifies the image and searches the internet for matching images, and finally returns 
results to a portable device. Boncyk et al. (US 7,016,532) teaches an image capture 
and identification system, which takes a picture, sends information to a database which 
returns best match in URL form. Wiedenhoefer (US 6,786,405) teaches a wireless 
device that communicates with remote sites and also has the capability of capturing 
images. 

1 0. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Scott Egan whose telephone number is (571) 270-1452. 
The examiner can normally be reached on Monday-Friday 8:00 a.m. - 5:00 p.m., EST 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Ngoc-Yen Vu can be reached on (571) 272-7320. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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